A Study of the Relationship Between Decision Model Naturalness and Performance

Two objectives in the design of decision support systems (DSS) are to improve decision-malting performance and to use DSS modeling forms that are natural, that is, to adopt modeling paradigms that are congruent with decision makers’ conceptual models of decision tasks. By accomplishing the latter objective, a DSS should enjoy better conceptual ease of use and face validity. However, past research finds that DSS deemed natural for a task by decision makers, DSS designers, and researchers alike, often do not improve (or even hinder) performance; the inverse also occurs. Further, decision-making behavior seems quite sensitive to minor task differences. How reliably are decision model naturalness and performance related? This study utilizes the bootstrapping paradigm of psychological research to help answer this question. In assessing the naturalness and performance of differing model paradigms over time and across levels oftask complexity, no single, systematic pattern emerges. But the results suggest that naturalness and performance are differentially sensitive to task contingencies. For example, while relative performance is stable over time only in the low complexity condition, relative naturalness is stable over time only in the intermediate complexity condition. One implication of the results is that conceptual ease of use maybe an unreliable predictor of a DSS’s effect on performance. DSS mechanisms may help decision makers better analyze model naturalness and performance.